Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning

In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks. To achieve this, we figure out a way to perturb affine transformations of neurons, and loss functions used in deep neural networks. In addition, our mechanism intentionally adds "more noise" into features which are "less relevant" to the model output, and vice-versa. Our theoretical analysis further derives the sensitivities and error bounds of our mechanism. Rigorous experiments conducted on MNIST and CIFAR-10 datasets show that our mechanism is highly effective and outperforms existing solutions.Comment: IEEE ICDM 2017 - regular pape

Background prior-based salient object detection via deep reconstruction residual

Detection of salient objects from images is gaining increasing research interest in recent years as it can substantially facilitate a wide range of content-based multimedia applications. Based on the assumption that foreground salient regions are distinctive within a certain context, most conventional approaches rely on a number of hand designed features and their distinctiveness measured using local or global contrast. Although these approaches have shown effective in dealing with simple images, their limited capability may cause difficulties when dealing with more complicated images. This paper proposes a novel framework for saliency detection by first modeling the background and then separating salient objects from the background. We develop stacked denoising autoencoders with deep learning architectures to model the background where latent patterns are explored and more powerful representations of data are learnt in an unsupervised and bottom up manner. Afterwards, we formulate the separation of salient objects from the background as a problem of measuring reconstruction residuals of deep autoencoders. Comprehensive evaluations on three benchmark datasets and comparisons with 9 state-of-the-art algorithms demonstrate the superiority of the proposed work

A Comparative Study of Theoretical Graph Models for Characterizing Structural Networks of Human Brain

Previous studies have investigated both structural and functional brain networks via graph-theoretical methods. However, there is an important issue that has not been adequately discussed before: what is the optimal theoretical graph model for describing the structural networks of human brain? In this paper, we perform a comparative study to address this problem. Firstly, large-scale cortical regions of interest (ROIs) are localized by recently developed and validated brain reference system named Dense Individualized Common Connectivity-based Cortical Landmarks (DICCCOL) to address the limitations in the identification of the brain network ROIs in previous studies. Then, we construct structural brain networks based on diffusion tensor imaging (DTI) data. Afterwards, the global and local graph properties of the constructed structural brain networks are measured using the state-of-the-art graph analysis algorithms and tools and are further compared with seven popular theoretical graph models. In addition, we compare the topological properties between two graph models, namely, stickiness-index-based model (STICKY) and scale-free gene duplication model (SF-GD), that have higher similarity with the real structural brain networks in terms of global and local graph properties. Our experimental results suggest that among the seven theoretical graph models compared in this study, STICKY and SF-GD models have better performances in characterizing the structural human brain network

Network-selective vulnerability of the human cerebellum to Alzheimer’s disease and frontotemporal dementia

Neurodegenerative diseases are associated with distinct and distributed patterns of atrophy in the cerebral cortex. Emerging evidence suggests that these atrophy patterns resemble intrinsic connectivity networks in the healthy brain, supporting the network-based degeneration framework where neuropathology spreads across connectivity networks. An intriguing yet untested possibility is that the cerebellar circuits, which share extensive connections with the cerebral cortex, could be selectively targeted by major neurodegenerative diseases. Here we examined the structural atrophy in the cerebellum across common types of neurodegenerative diseases, and characterized the functional connectivity patterns of these cerebellar atrophy regions. Our results showed that Alzheimer’s disease and frontotemporal dementia are associated with distinct and circumscribed atrophy in the cerebellum. These cerebellar atrophied regions share robust and selective intrinsic connectivity with the atrophied regions in the cerebral cortex. These findings for the first time demonstrated the selective vulnerability of the cerebellum to common neurodegenerative disease, extending the network-based degeneration framework to the cerebellum. Our work also has direct implications on the cerebellar contribution to the cognitive and affective processes that are compromised in neurodegeneration as well as the practice of using the cerebellum as reference region for ligand neuroimaging studies

Coupling Artificial Neurons in BERT and Biological Neurons in the Human Brain

Linking computational natural language processing (NLP) models and neural responses to language in the human brain on the one hand facilitates the effort towards disentangling the neural representations underpinning language perception, on the other hand provides neurolinguistics evidence to evaluate and improve NLP models. Mappings of an NLP model's representations of and the brain activities evoked by linguistic input are typically deployed to reveal this symbiosis. However, two critical problems limit its advancement: 1) The model's representations (artificial neurons, ANs) rely on layer-level embeddings and thus lack fine-granularity; 2) The brain activities (biological neurons, BNs) are limited to neural recordings of isolated cortical unit (i.e., voxel/region) and thus lack integrations and interactions among brain functions. To address those problems, in this study, we 1) define ANs with fine-granularity in transformer-based NLP models (BERT in this study) and measure their temporal activations to input text sequences; 2) define BNs as functional brain networks (FBNs) extracted from functional magnetic resonance imaging (fMRI) data to capture functional interactions in the brain; 3) couple ANs and BNs by maximizing the synchronization of their temporal activations. Our experimental results demonstrate 1) The activations of ANs and BNs are significantly synchronized; 2) the ANs carry meaningful linguistic/semantic information and anchor to their BN signatures; 3) the anchored BNs are interpretable in a neurolinguistic context. Overall, our study introduces a novel, general, and effective framework to link transformer-based NLP models and neural activities in response to language and may provide novel insights for future studies such as brain-inspired evaluation and development of NLP models

Characterization of a human tumorsphere glioma orthotopic model using magnetic resonance imaging

Magnetic resonance imaging (MRI) is the imaging modality of choice by which to monitor patient gliomas and treatment effects, and has been applied to murine models of glioma. However, a major obstacle to the development of effective glioma therapeutics has been that widely used animal models of glioma have not accurately recapitulated the morphological heterogeneity and invasive nature of this very lethal human cancer. This deficiency is being alleviated somewhat as more representative models are being developed, but there is still a clear need for relevant yet practical models that are well-characterized in terms of their MRI features. Hence we sought to chronicle the MRI profile of a recently developed, comparatively straightforward human tumor stem cell (hTSC) derived glioma model in mice using conventional MRI methods. This model reproduces the salient features of gliomas in humans, including florid neoangiogenesis and aggressive invasion of normal brain. Accordingly, the variable, invasive morphology of hTSC gliomas visualized on MRI duplicated that seen in patients, and it differed considerably from the widely used U87 glioma model that does not invade normal brain. After several weeks of tumor growth the hTSC model exhibited an MRI contrast enhancing phenotype having variable intensity and an irregular shape, which mimicked the heterogeneous appearance observed with human glioma patients. The MRI findings reported here support the use of the hTSC glioma xenograft model combined with MRI, as a test platform for assessing candidate therapeutics for glioma, and for developing novel MR methods

When Brain-inspired AI Meets AGI

Artificial General Intelligence (AGI) has been a long-standing goal of humanity, with the aim of creating machines capable of performing any intellectual task that humans can do. To achieve this, AGI researchers draw inspiration from the human brain and seek to replicate its principles in intelligent machines. Brain-inspired artificial intelligence is a field that has emerged from this endeavor, combining insights from neuroscience, psychology, and computer science to develop more efficient and powerful AI systems. In this article, we provide a comprehensive overview of brain-inspired AI from the perspective of AGI. We begin with the current progress in brain-inspired AI and its extensive connection with AGI. We then cover the important characteristics for both human intelligence and AGI (e.g., scaling, multimodality, and reasoning). We discuss important technologies toward achieving AGI in current AI systems, such as in-context learning and prompt tuning. We also investigate the evolution of AGI systems from both algorithmic and infrastructural perspectives. Finally, we explore the limitations and future of AGI

Characteristics of Envelope Genes in a Chinese Chronically HIV-1 Infected Patient With Broadly Neutralizing Activity

Exploring the characteristics of the HIV-1 envelope glycoprotein (env) gene in a natural HIV-1 infected individual, with broadly neutralizing activity, may provide insight into the generation of such broadly neutralizing antibodies and initiate the design of an appropriate immunogen. Recently, a chronically HIV-1 infected patient with broadly neutralization activity was identified and a VRC01-class neutralizing antibody DRVIA7 (A7) was isolated from the patient. In the present study, 155 full length HIV-1 env gene fragments (including 68 functionally Env clones) were amplified longitudinally from the plasma of six time points spanning over 5 years in this donor. Viral features were analyzed by comparing Env clones of different time points, as well as 165 Chinese HIV-1 subtype B env sequences from HIV Sequence Database (Chinese B_database). Shorter V1 length, less potential glycan and a lower ratio of NXT: NXS in gp160 were observed in the first five time points compared to that from the last time points, as well that from the Chinese B_database. A sequence analysis and a neutralization assay of Env-pseudoviruses showed that the increasing diversity of env sequences in the patient was consistent with the appearance and maturation of A7 lineage antibodies. The potent neutralization activity and viruses that escaped from the neutralization of the concurrent autologous plasma, are consistent with higher residue variations at the antibody recognition sites. Almost all viruses from the plasmas were neutralization-resistant to VRC01 and A7 lineage antibodies. For a chronically HIV-1 infected individual over 10 years, we found that greater viral diversity, short V1 sequences and less potential N-linked glycosylation (PNGS) in V1, might be associated with the development of broadly neutralizing antibody responses

Fine-Granularity Functional Interaction Signatures for Characterization of Brain Conditions

In the human brain, functional activity occurs at multiple spatial scales. Current studies on functional brain networks and their alterations in brain diseases via resting-state functional magnetic resonance imaging (rs-fMRI) are generally either at local scale (regionally confined analysis and inter-regional functional connectivity analysis) or at global scale (graph theoretic analysis). In contrast, inferring functional interaction at fine-granularity sub-network scale has not been adequately explored yet. Here our hypothesis is that functional interaction measured at fine-granularity subnetwork scale can provide new insight into the neural mechanisms of neurological and psychological conditions, thus offering complementary information for healthy and diseased population classification. In this paper, we derived fine-granularity functional interaction (FGFI) signatures in subjects with Mild Cognitive Impairment (MCI) and Schizophrenia by diffusion tensor imaging (DTI) and rsfMRI, and used patient-control classification experiments to evaluate the distinctiveness of the derived FGFI features. Our experimental results have shown that the FGFI features alone can achieve comparable classification performance compared with the commonly used inter-regional connectivity features. However, the classification performance can be substantially improved when FGFI features and inter-regional connectivity features are integrated, suggesting the complementary information achieved from the FGFI signatures

Anatomy-Guided Dense Individualized and Common Connectivity-Based Cortical Landmarks (A-DICCCOL)

Establishment of structural and functional correspondences of human brain that can be quantitatively encoded and reproduced across different subjects and populations is one of the key issues in brain mapping. As an attempt to address this challenge, our recently developed Dense Individualized and Common Connectivity-based Cortical Landmarks (DICCCOL) system reported 358 connectional landmarks, each of which possesses consistent DTI-derived white matter fiber connection pattern that is reproducible in over 240 healthy brains. However, the DICCCOL system can be substantially improved by integrating anatomical and morphological information during landmark initialization and optimization procedures. In this paper, we present a novel anatomy-guided landmark discovery framework that defines and optimizes landmarks via integrating rich anatomical, morphological, and fiber connectional information for landmark initialization, group-wise optimization and prediction, which are formulated and solved as an energy minimization problem. The framework finally determined 555 consistent connectional landmarks. Validation studies demonstrated that the 555 landmarks are reproducible, predictable, and exhibited reasonably accurate anatomical, connectional, and functional correspondences across individuals and populations and thus are named anatomy-guided DICCCOL or A-DICCCOL. This A-DICCCOL system represents common cortical architectures with anatomical, connectional, and functional correspondences across different subjects and would potentially provide opportunities for various applications in brain science